In order to solve the problem of low accuracy of some distortion types judgment by image single distortion type judgment algorithm, an image single distortion type judgment method based on two-channel Convolutional Neural Network (CNN) was proposed. Firstly, the fixed size image block was obtained by cropping the image, and the high-frequency information map was obtained by Haar wavelet transform of the image block. Then, the image block and the corresponding high-frequency information map were respectively input into the convolutional layers of different channels to extract the deep feature map, and the deep features were fused and input into the fully connected layer. Finally, the values of the last layer of the fully connected layer were input into the Softmax function classifier to obtain the probability distribution of the single distortion type of the image. Experimental results on LIVE database show that, the proposed method has the image single distortion type judgement accuracy up to 95.21%, and compared with five other image single distortion type judgment methods for comparison, the proposed method has the accuracies for judging JPEG2000 and fast fading distortions improved by at least 6.69 percentage points and 2.46 percentage points respectively. The proposed method can accurately identify the single distortion type in the image.